Display method and system for enabling an operator to visualize and correct alignment errors in imaged data sets

ABSTRACT

A method to visualize and correct alignment errors between paired 2D and 3D data sets is described. In a representative embodiment, a display interface used for dental implant planning includes one or more display areas that enable the operator to visualize alignment errors between the paired 2D and 3D data sets. A first display area renders 3D cone beam data. A second display area renders one or more (and preferably three (3) mutually orthogonal views) slices of the cone beam data. A third display area displays a view of a 2D scanned surface map (obtained from an intra-oral scan, or the scan of a model). According to a first aspect, the view of the 2D scanned surface map in the third display area is “textured” by coloring the 2D surface model based on the intensity of each 3D pixel (or “voxel”) that it intersects.

BACKGROUND OF THE INVENTION Technical Field

This disclosure relates generally to computer-assisted techniques forcreating dental restorations.

Brief Description of the Related Art

The art of fabricating custom-fit prosthetics in the dental field iswell-known. Prosthetics are replacements for tooth or bone structure.They include restorations, replacements, inlays, onlays, veneers, fulland partial crowns, bridges, implants, posts, and the like. Typically, adentist prepares a tooth for a restoration by removing existing anatomy,which is then lost. The resultant prepared area (a “preparation”) isthen digitized (or, in the alternative, a dental impression is taken)for the purpose of constructing a restoration, appliance orsubstructure. The restoration itself may be constructed through avariety of techniques including manually constructing the restoration,using automated techniques based on computer algorithms, or acombination of manual and automated techniques.

Computer-assisted techniques have been developed to generatethree-dimensional (“3D”) visual images of physical objects, such as adental preparation. In general, the 3D image may be generated by acomputer that processes data representing the surfaces and contours of aphysical object. The computer displays the 3D image on a screen or acomputer monitor. The computer typically includes a graphical userinterface (GUI). Data is generated by optically scanning the physicalobject and detecting or capturing the light reflected off of the object.Based on processing techniques, the shape, surfaces and/or contours ofthe object may be modeled by the computer. During the process ofcreating a tooth restoration model, one or more user interface tools maybe provided to facilitate the design process. One known displaytechnique uses a computer monitor that, under software control, displaysa 3-dimensional representation of a tooth model.

It is also known in the art to use such computer-aided design systems tofacilitate the production of a crown to be placed on a custom implantabutment. Because the implant abutment is custom designed (i.e., to fitthe implant), the interior of the crown that attaches to the abutmentalso needs to be custom designed for the particular case. The usualprocess followed is for an implant to be inserted into the jawbone (ormaxillary-upper arch) of a patient. An abutment (made, for example, fromtitanium or zirconia) is then screwed (or placed or cemented) onto thetop of the implant and is then adjusted by the dentist using dentaltools. At this point, the abutment may be digitized by a 3D scanner, anda crown model generated using CAD techniques, and finally a physicalcrown (or appliance) milled out of a dental material such as ceramic,composite or metal. The abutment is scanned at the time it is customized(placed), i.e., at the time that the implant is first inserted. Whencustomization of the implant is completed, either the abutment isremoved for scanning outside the mouth, or the abutment is scannedinside the mouth while attached to the implant. U.S. Publication No.20090087817, assigned to D4D Technologies, LLC, describes this approach.

Helical (or spiral) cone beam computed tomography (CBCT) is a knowntechnique for three-dimensional (3D) computer tomography in which asource (typically X-rays) describes a helical trajectory relative to anobject being scanned while a two-dimensional (2D) array of detectorsmeasures the transmitted radiation on part of a cone of rays emanatingfrom the source. Cone beam 3-D dental imaging systems provide dentistsand specialists with high-resolution volumetric images of a patient'smouth, face and jaw areas. Three-dimensional views of all oral andmaxillofacial structures allows for more thorough analysis of bonestructures and tooth orientation to optimize implant treatment andplacement, selection of the most suitable implant type and angulationsprior to surgery. Representative commercial systems implementing thesetechnologies are the i-CAT from Imaging Sciences International, Inc. andthe GXCB-500 (powered by i-CAT) from Gendex Dental Systems, Inc.

Thus, it is well-known to use different imaging modalities (e.g., CT,MRI, OCT, ultrasound, microscopy, camera-based, etc.) to produce imagesfor dental CAD CAM systems. In one use case, such as the planning forimplant surgery, a first modality may be used to produce a scannedsurface map of a patient's existing dentition, and a second modality maybe used to produce 3D cone beam computed tomography data. Of course,each of the data sets provides different information, and it isdesirable to provide a mechanism to align such data. While there areknown techniques for this purpose, such techniques often produce lessthan optimal results. Errors in alignment may manifest themselves asdistortions in an end result. For example, an alignment error betweenthe 2D surface map and the 3D computed tomography data may result ininaccurate positioning of a dental implant device with respect tolandmarks in either or both data sets. Moreover, because the data setstypically are acquired from significantly different imaging modalities,the characteristics of the data may be quite different, whichexacerbates the problem for an automated solution. Due to these issues,most automated and semi-automated approaches to align such data sets mayyield significant alignment errors. As a result, there is a need toprovide a robust mechanism to allow users to visualize alignment errors,and to manually or automatically correct them.

BRIEF SUMMARY

A method to visualize and correct alignment errors between paired 2D and3D data sets is described. In a representative embodiment, a displayinterface includes one or more display areas that enable the operator tovisualize alignment errors between the paired 2D and 3D data sets. Afirst display area renders 3D cone beam data. A second display arearenders one or more (and preferably three (3) mutually orthogonal views)slices of the cone beam data. A third display area displays a view of a2D scanned surface map (obtained from an intra-oral scan, or the scan ofa model). According to a first aspect of this disclosure, the view ofthe 2D scanned surface map in the third display area is “textured” bycoloring the 2D surface model based on the intensity of each 3D pixel(or “voxel”) that it intersects.

Although not meant to limit this disclosure, the visualization techniquedescribed above may be implemented within a dental implant planningsoftware system used to plan and design an implant abutment to support arestoration.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter herein may be better understood with reference to thefollowing drawings and its accompanying description. Unless otherwisestated, the components in the figures are not necessarily to scale,emphasis instead being placed upon illustrating the principles of theinvention. Moreover, in the figures, like referenced numerals designatecorresponding parts throughout the different views.

FIG. 1 illustrates an interface displaying 3D cone beam data;

FIG. 2 illustrates an interface displaying a 2D surface model generatedby an intra-oral scan;

FIG. 3 illustrates an interface showing a pairing of the 3D and 2Dscans;

FIG. 4 illustrates an interface according to this invention; and

FIG. 5 illustrates a computer system in which the method describedherein may be implemented.

DETAILED DESCRIPTION

As noted above, this disclosure provides a display method, preferablyimplemented in a computer, such as a workstation. For illustratedpurposes, the workstation is a single machine, but this is not alimitation. More generally, the method is implemented using one or morecomputing-related entities (systems, machines, processes, programs,libraries, functions, code, or the like) that facilitate or provide theinventive functionality. A representative machine is a computer runningcommodity hardware, an operating system, an application runtimeenvironment, and a set of applications or processes (e.g., linkablelibraries, native code, or the like, depending on platform), thatprovide the functionality of a given system or subsystem. The inventionmay be implemented in a standalone machine, or across a distributed setof machines.

In a representative embodiment, a computer workstation in which theinvention is implemented comprises hardware, suitable storage and memoryfor storing an operating system, one or more software applications anddata, conventional input and output devices (a display, a keyboard, apoint-and-click device, and the like), other devices to provide networkconnectivity, and the like. An intra-oral digitizer wand is associatedwith the workstation to obtain optical scans from a patient's anatomy.The digitizer scans the restoration site with a scanning laser systemand delivers live images to a monitor on the workstation. An intra-oraldigital (IOD) scanner and associated computer-aided design system thatmay be used for this purpose is the E4D Dentist™ system, manufactured byD4D Technologies, LLC. The E4D Dentist system is a comprehensivechairside CAD CAM system that produces inlays, onlays, full crowns andveneers. A handheld laser scanner in the system captures a true 3-Dimage either intraorally, from impressions or from models. Designsoftware (e.g., DentaLogic™) in this system is used to create a 3-Dvirtual model.

Generalizing, a display interface according to this disclosure isgenerated in software (e.g., a set of computer program instructions)executable in at least one processor. A representative implementation iscomputer program product comprising a tangible medium on which givencomputer code is written, stored or otherwise embedded. The displayinterface comprises an ordered set of display tabs and associateddisplay panels or “viewports.” Although the illustrative embodimentshows data sets displayed within multiple viewports on a single display,this is not a limitation, as the various views may be displayed usingmultiple windows, views, viewports, and the like. The display interfacemay be web-based, in which case the views of displayed asmarkup-language pages. The interface exposes conventional displayobjects such as tabbed views, pull-down menus, browse objects, and thelike.

The system also receives cone beam data for a cone beam data source. Asnoted above, representative commercial systems that implement cone beamscanning are the i-CAT from Imaging Sciences International, Inc. and theGXCB-500 (powered by i-CAT) from Gendex Dental Systems, Inc. These aremerely representative, as any convenient source of cone beam data may beused. Moreover, while cone beam data is preferred as the source of the3D scan, this is not a limitation, as any 3D data source may be used.

FIG. 1 illustrates several views of a 3D cone beam data displayed on aninterface 101, whereas FIG. 2 represents the display interface 201 forthe 2D scanned surface map data. In this embodiment, FIG. 2 illustratesthe known user interface from the E4D Dentist system available from D4DTechnologies, LLC and described by commonly-owned, U.S. Pat. No.7,184,150, the disclosure of which is incorporated by reference. As iswell-known, using that system the prepared area and adjacent teeth arescanned using the digitizer, and a 3D model of the prepared area isobtained. This information is then used to produce a 3D model of adesired restoration. Such a process can be performed using the DesignCenter available as part of the E4D Dentist system. FIG. 3 illustrateshow the 3D and 2D data sets are paired, preferably in a singleside-by-side view 300 (although any other convenient orientation may beimplemented). In this dual view, the operator simply selects and clickson what he or she considers are common points on the cone beam data andthe scanned data. Typically, three (3) common points on each data setare sufficient to automatically align the data sets. The algorithms andtechniques for performing the data set alignment are not part of thisdisclosure, as there are known techniques and technologies that areuseful for this purpose. Any alignment method may be used.

For example, standard methods to align 2D and 3D may include extractinga set of edge points from the 3D data, based in part on characteristicsof the 3D data, and computing a transformation that minimizes a costfunction based on the sum of the distances between the closest points inthe transformed data sets. The computation of the optimal transformationmay be iterated until convergence is achieved. An alternative approachmay include identifying corresponding feature sets on both 2D and 3Ddata sets, and computing a transformation that minimizes a cost functionbased on the sum of distances between the corresponding points.Corresponding feature sets may include regions of high curvature, orspecific shapes or structures present in both 2D and 3D data sets.

FIG. 4 illustrates the unique display method according to thisdisclosure. It is assumed that the data sets have been aligned in themanner described above. In a representative embodiment, a displayinterface 400 includes one or more display areas that enable theoperator to visualize and correct alignment errors between the paired 2Dand 3D data sets. A display area 404 renders one or more (and preferablythree (3) mutually orthogonal views) slices of the cone beam data. Thus,there is a first display slice 404 a, a second display slice 404 b, anda third display slice 404 c. While all three views are not required,they are preferred. A main display area 406 displays a view of a 2Dscanned surface map (obtained from an intra-oral scan, or the scan of amodel). According to a first aspect, the view of the 2D scanned surfacemap in the third display area is “textured” by coloring the 2D surfacemodel based on the intensity of each 3D pixel (or “voxel”) that itintersects. In view 406, the intensities of the cone beam computedtomography data are overlaid onto the 2D surface of the (in this case)intra-oral scan. In this example, the soft tissue 408 is colored with alower intensity as compared to the teeth structures 410, and this isbecause the soft tissue has a lower density than that of the teeth.Thus, the textured view provides the operator with a unique perspectiveto conform alignment using the visualized hard and soft tissuelandmarks. In addition, and according to a second aspect, each of thepreferably mutually orthogonal cone view data slices (in viewports 404a-c) is overlaid with a wireframe projection or model from the 2Dsurface model data. The wireframe projection is shown in the darker (orbold) lines in each of the separate viewports. Preferably, the operatorcan rotate and move the 2D surface model in any view by clicking anddragging with a data entry device (a mouse or keyboard). Thevisualizations update, preferably in real-time, as one data set istransformed with respect to the other.

Although not shown in FIG. 4, the display interface may also include adisplay area for the 3D cone beam data.

Although it is preferred to display the lower density structures withcolors of lower intensity, this is not a limitation, as the oppositeapproach may be used (i.e. displaying lower density structures withcolors of higher intensity). Also, in lieu of using color variations,other display constructs (such as shading, symbols, text, numbers, etc.)may be used to illustrate the “texturing.”

As used herein, the phrase “display area” should be broadly construed torefer to a window, a sub-window, a display tab, a portion of a displayarea, a viewport, and the like.

Thus, preferably the textured view is formed by coloring (or shading, orotherwise providing a visual indication on) the surface of the 2D dataas a function of the intensity of the spatial points (voxels) at whichit intersects the 3D data set. In addition, preferably each slice of the3D data set (and preferably there are three such mutually orthogonalslices) is superposed with a wireframe projection onto that same sliceof the aligned 2D data set. This user interface allows the operator toeasily identify and correct any alignment errors, as the interfaceenables the operator to rigidly transform either data set with respectto the other data set, wherein the underlying alignment routine updateand adjust the visualizations accordingly. When the alignment isaccurate, the contrast at the gum line of the teeth in the 2D scan ishigh, because teeth and soft tissue exhibit different intensities in thecone beam data. If the alignment is poor, the change in contrast occursaway from the gum line, making errors in alignment easy to detect andcorrect. In particular, the operator can use conventional displayinterface tools to manually manipulate the alignment, e.g., by selectingone of the slice views and dragging the associated wireframe projectionto rotate and translate the associated alignment matrix, which updatesautomatically.

Although not meant to be limiting, the technique described above may beimplemented within a dental implant planning software package andsystem.

Several of the processing steps are performed in a computer. As seen inFIG. 5, a representative computer 100 comprises hardware 102, suitablestorage 104 and memory 105 for storing an operating system 106, one ormore software applications 108 and data 110, conventional input andoutput devices (a display 112, a keyboard 114, a point-and-click device116, and the like), other devices 118 to provide network connectivity,and the like. A laser digitizer system 115 is used to obtain opticalscans from a patient's dental anatomy. Using a conventional graphicaluser interface 120, an operator can view and manipulate scannedinformation and models as they are rendered on the display 112.

While the above describes a particular order of operations performed bycertain embodiments of the invention, it should be understood that suchorder is exemplary, as alternative embodiments may perform theoperations in a different order, combine certain operations, overlapcertain operations, or the like. References in the specification to agiven embodiment indicate that the embodiment described may include aparticular feature, structure, or characteristic, but every embodimentmay not necessarily include the particular feature, structure, orcharacteristic. Further, while given components of the system have beendescribed separately, one of ordinary skill will appreciate that some ofthe functions may be combined or shared in given systems, machines,devices, processes, instructions, program sequences, code portions, andthe like.

Having described our invention, what we claim is as follows.

The invention claimed is:
 1. A method of visualizing and correctingalignment errors between 2D and 3D data sets in an implant planningtool, wherein the 3D data set comprises one or more spatial points eachhaving an intensity, comprising: displaying a textured view that isgenerated by providing a given display indication on a surface of the 2Ddata set as a function of intensities of one or more spatial points ofthe 3D data set at which the surface intersects the 3D data set, wherethe textured view is generated in software executing in a hardwareelement responsive to receipt of operator-entered data indicating one ormore common points on the 2D data set and the 3D data set andillustrates an alignment between the 2D and 3D data sets; in associationwith the textured view, displaying one or more slices of the 3D dataset, wherein each of the slices of the 3D data set is overlaid with awireframe projection from the 2D data set to form a visualization; andresponsive to receipt of operator-entered data indicating a rotation ormovement of the 2D data set, updating the visualizations as the 2D and3D data sets are transformed relative to each other.
 2. The method asdescribed in claim 1 wherein the 2D data set is a surface map generatedfrom a surface scan, and wherein the 3D data set is volume datagenerated from a cone beam scan.
 3. The method as described in claim 1wherein the given display indication is a coloration.
 4. The method asdescribed in claim 1 wherein the slices of the 3D set aremutually-orthogonal, and wherein each slice is displayed in a dedicateddisplay area.
 5. An apparatus for use in visualizing and correctingalignment errors between 2D and 3D data sets in an implant planningtool, wherein a 3D data set comprises one or more spatial points eachhaving an intensity, comprising: a processor; computer memory storingcomputer program instructions executed by the processor to performoperations comprising: displaying a textured view that is generated byproviding a given display indication on a surface of the 2D data set asa function of intensities of one or more spatial points of the 3D dataset at which the surface intersects the 3D data set, where the texturedview is generated in response to receipt of operator-entered dataindicating one or more common points on the 2D data set and the 3D dataset and illustrates an alignment between the 2D and 3D data sets; inassociation with the textured view, displaying one or more slices of the3D data set, wherein each of the slices of the 3D data set is overlaidwith a wireframe projection from the 2D data set to form avisualization; and responsive to receipt of operator-entered dataindicating a rotation or movement of the 2D data set, updating thevisualizations as the 2D and 3D data sets are transformed relative toeach other.
 6. The apparatus as described in claim 5 wherein the 2D dataset is a surface map generated from a surface scan, and wherein the 3Ddata set is volume data generated from a cone beam scan.
 7. Theapparatus as described in claim 5 wherein the given display indicationis a coloration.
 8. The apparatus as described in claim 5 wherein theslices of the 3D data set are mutually-orthogonal, and wherein eachslice is displayed in a dedicated display area.
 9. An article comprisinga non-transitory tangible machine-readable medium that stores a program,wherein a 3D data set comprises one or more spatial points each havingan intensity, the program being executed by a machine to performoperations comprising: displaying, on a display device associated withthe machine, a textured view that is generated by providing a givendisplay indication on a surface of a 2D data set as a function ofintensities of one or more spatial points of the 3D data set at whichthe surface intersects a 3D data set, where the textured view isgenerated in response to receipt of operator-entered data indicating oneor more common points on the 2D data set and the 3D data set andillustrates an alignment between the 2D and 3D data sets; in associationwith the textured view, displaying one or more slices of the 3D dataset, wherein each of the slices of the 3D data set is overlaid with awireframe projection from the 2D data set to form a visualization; andresponsive to receipt of operator-entered data indicating a rotation ormovement of the 2D data set, updating the visualizations as the 2D and3D data sets are transformed relative to each other.
 10. The article asdescribed in claim 9 wherein the 2D data set is a surface map generatedfrom a surface scan, and wherein the 3D data set is volume datagenerated from a cone beam scan.
 11. The article as described in claim 9wherein the given display indication is a coloration.
 12. The article asdescribed in claim 9 wherein the slices of the 3D data set aremutually-orthogonal, and wherein each slice is displayed in a dedicateddisplay area.